HIV self-test performance evaluation among priority populations in rural Mozambique: Results from a community-based observational study

Background In 2021, Mozambique initiated community-based oral HIV self-testing (HIVST) to increase testing access and uptake among priority groups, including adult males, adolescents, and young adults. Within an HIVST pilot project, we conducted a performance evaluation assessing participants’ ability to successfully conduct HIVST procedures and interpret results. Methods A cross-sectional study was performed between February-March 2021 among employees, students (18–24 years of age), and community members, using convenience sampling, in two rural districts of Zambézia Province, Mozambique. We quantified how well untrained users performed procedures for the oral HIVST (Oraquick®) through direct observation using a structured checklist, from which we calculated an HIVST usability index (scores ranging 0–100%). Additionally, participants interpreted three previously processed anonymous HIVST results. False reactive and false non-reactive interpretation results were presented as proportions. Bivariate analysis was conducted using Chi-square and Fisher exact tests. Results A total of 312 persons participated (131[42%] community members, 71[23%] students, 110[35%] employees); 239 (77%) were male; the mean age was 28 years (standard deviation 10). Average usability index scores were 80% among employees, 86% among students, and 77% among community members. Main procedural errors observed included “incorrect tube positioning” (49%), “incorrect specimen collection” (43%), and “improper waiting time for result interpretation” (42%). From the presented anonymous HIVST results, 75% (n = 234) correctly interpreted all three results, while 9 (3%) of study participants failed to correctly interpret any results. Overall, 36 (12%) gave a false non-reactive result interpretation, 21 (7%) a false reactive result interpretation, and 14 (4%) gave both false non-reactive and false reactive result interpretations. Community members generally had lower performance. Conclusions Despite some observed testing procedural errors, most users could successfully perform an HIVST. Educational sessions at strategic places (e.g., schools, workplaces), and support via social media and hotlines, may improve HIVST performance quality, reducing the risk of incorrect interpretation.


Introduction
HIV self-testing (HIVST) is a strategy recommended by the World Health Organization (WHO) to enhance HIV testing uptake rates [1,2], especially among groups that are traditionally more difficult to reach.HIVST can be performed assisted or unassisted, however, it is necessary to seek services to confirm a reactive HIVST result, as a single rapid test (i.e., HIVST) is not sufficient to make an HIV positive diagnosis.Studies evaluating HIVST performance among specific priority populations such as female sex workers (FSW), as well as the general population, including males and young adults are usually country-specific [3][4][5][6][7][8].Variability in HIVST performance has been observed based on the HIVST sample type, namely, whole blood versus oral saliva.In the past, slightly poorer test performance was seen with oral salivabased samples, with a lower sensitivity and lower positive predictive value (PPV) when evaluated in low prevalence settings [9].However, subsequent studies performed in high HIV prevalence settings have now shown comparable test performance with oral saliva and whole blood based samples, with excellent PPV's being reported [10,11].It also is important to continuously monitor HIVST performance in order to tailor information, educational needs, and communication strategies at the community level.
Mozambique, a geographically large country with an estimated 65% of the population living in rural areas, has an HIV prevalence of 12.5%, according to the most recent national HIV impact assessment in 2021 [12].The 2015 national HIV survey showed that 39% of women and 60% of men had never been tested for HIV [13].To increase HIV testing coverage, the country initiated an HIVST pilot project in one province in the central region of the country in 2019 (Zambe ´zia Province); results have been published elsewhere [14].Following the pilot, the Ministry of Health (MOH) elaborated national HIVST guidelines and began expanding its implementation in 2021 [15].The MOH further piloted community distribution of HIVST test kits in 2021 and expanded the availability of this testing approach for the general population nationally beginning in 2022, in order to reach those who are generally difficult to reach.
The national guidelines recommend confirmation of a reactive HIVST at the health facility.No data exist yet on the quality of HIVST performance in the country.The aim of this study was to evaluate the quality of HIVST performance and result interpretation among employees (who are primarily male in our context), young adults (students), and community members in general, who were each identified as priority groups for HIVST by the MOH.

Study setting
The study was implemented in Zambe ´zia Province, which is situated in the central region of Mozambique and whose large population (~6 million) is mainly rural (82%) [16].In 2021, based on the recently completed Population-Based HIV Impact Assessment, the adult (� 15 years of age) HIV prevalence in Zambe ´zia was 17.1% (95% CI 13.0%-21.2%)[12].Alto Molo ´cuè and Mocuba, two rural districts in Zambe ´zia, where the HIVST pilot study was implemented, were selected for inclusion in this evaluation.The study populations were recruited from secondary/ superior level schools, small companies in the two districts, and from communities surrounding the two main district health facilities.All schools and companies in the two districts were listed and approached, and those interested in the project were included in the study, resulting in inclusion of 3 schools and 5 companies.

Study design and study population
We used a cross-sectional study design and recruited participants from the following groups: employees, students and community members.Inclusion criteria were: 1) for all participants: willingness to know their HIV status and a minimum age of 18 years; 2) for employees: adult, salaried employee for at least one month at a registered company in the select districts; 3) for students: 18 to 24 years of age and studying in one of the recognized tertiary educational institutions in the selected districts; 4) for community members: individuals living in the selected districts (who do not otherwise qualify as an "employee" or "student").Individuals with a selfreported known HIV-positive status were excluded.The groups were selected to capture different populations: males (employees), young adults (students), and the general population (community members).

Data collection
Data were collected between February and March 2021.The research staff convened group information sessions for managers of the selected companies and directors of selected schools and institutions prior to the data collection activities to provide information about the study objectives and procedures; the managers and directors then informed their employees and students respectively regarding the study.For implementation in the community, community leaders were invited to participate in information sessions to learn more about the objectives and procedures of the study.Subsequently, the leaders disseminated information about the study within the community.Recruitment was done through convenience sampling, as per availability and interest on the date of data collection.Interested individuals were referred to designated places within respective recruitment locations, for example, a meeting room at the company locations, a classroom or meeting room at the schools, or a meeting room in a nearby health facility or school for community members.The research assistants explained about the study and completed the informed consent process with the interested individuals.Consenting participants were evaluated on their performance of HIVST procedures through observation by independently trained research assistants.The oral HIVST OraQuick HIV1/2 1 Self-Test (OraSure Technologies Inc., Bethlehem, PA, U.S.) was used through a donation from OraSure Technologies.This test is approved by the U.S. Food and Drug Administration as an oral home HIV test [17] and has been pre-qualified by the WHO [18].The test entails using an oral swab and no blood samples are needed.The manufacturers' written instructions were available to participants in Portuguese and a demonstration video of the procedures was offered in Portuguese and in the two main local languages, Chuabo, and Manhaua, for those interested.The video was adapted from the manufacturers' video (http://www.oraquick.com/Taking-the-Test/How-To-Video) to local context and language.Six trained research assistants watched how participants performed the HIV self-test, without interfering or responding to any question related to the testing procedures (one assistant per participant).The research assistants used a structured checklist with a total of 20 checklist items (13 procedure-related items, 7 supportive items) based on other performance evaluations and adapted for the Mozambican context for this study [3,6].The research assistant observed and documented the participant's performance of the testing procedures per the manufacturer's instructions on the checklist, including the completion (or not) of the four specific testing components: Preparation, Collection, Mixing, and Results Reading.In addition, participants were asked to interpret the results of three previously processed anonymous self-tests.The tests, with either an HIV-non-reactive, HIVreactive, or invalid results, were shown at random to the participants via photos, to assess the participants' interpretation of test results.Participants were not required to share their own test result after completing the HIVST and results were not seen by the research assistants.

Statistical considerations
Performance evaluation sample size was calculated based on pilot studies [8,19], in which performance without errors was observed 85% of the time.We anticipated that approximately 70% of employees, 85% of students, and 50% of community members would be able to perform the HIVST without errors.Assuming a margin of error (MOE) is equal to half of the Wilson Score interval at 90% confidence, and solving the following equation for n, we estimated that a sample size of at least 55 employees, 35 students, and 65 community members from each district would be sufficient to estimate the probability of success within each district with an MOE of 10%.
ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi Where z is the z-score corresponding to the desired confidence interval, p is the expected proportion of participants who would perform the HIVST without errors, and MOE is the desired margin of error.HIVST usability was calculated as the sum of the scores of each of the 13 observations, where the observations with expected negative ("No" as response) and observations with expected positive ("Yes" as response) responses were counted.The usability percentage for each observation was calculated and an average was provided for the general usability index (UI), ranging between 0% (not useable) to 100% (highly usable).False reactive and false non-reactive interpretations were calculated as follows: a) false reactive: when participants provided an interpretation of "reactive" to a test result that was actually non-reactive or invalid, and b) false non-reactive: when participants provided an interpretation of "non-reactive" to a test result that was actually reactive or invalid.A correct interpretation was defined as follows: providing a "reactive" interpretation when the test result was reactive, providing a "non-reactive" interpretation when the test result was non-reactive, and providing an "invalid" interpretation when the test result was invalid.The Chi-square (or Fisher exact) tests were used for comparison analyses among the three participant groups.Descriptive and comparative analyses were performed in R version 4.0.1 [20].

Ethical considerations
The protocol and instruments were approved by the institutional health ethics committee of the "Instituto Nacional
Refusal rate was 7%, mainly among community members, related to the individuals not feeling comfortable doing an HIV test as part of the study procedures, or not having enough time (Fig 1).Most participants were male (239, 77%).Among participants, 99 (32%) reported that Portuguese was their native language; 223 (71%) stated their reading abilities in Portuguese as "good" to "very good".The vast majority (83%) of participants reported that they had previously been tested for HIV, with approximately one fifth (19%) stating that their prior HIV testing took place more than two years prior to data collection (Table 1).).The main performance errors observed included incorrect tube positioning into the stand (152, 49%), incorrect specimen collection (134, 43%), and improper waiting time for result interpretation (130, 42%) (Table 2).

Interpretation
All participants received three anonymous, predefined HIVST results using photos and were asked to provide the interpretation of the result.A total of 936 test results were presented (330, 213, and 393 results presented to employees, students, and community members, respectively), with an overall correct interpretation rate of 88% (823/936).When disaggregated by group, community members had a lower correct interpretation rate (84%; 332/393), with students performing the best (95%; 203/213) (S2 Table ).When examining the results interpretation performance at the individual participant level, 75% (n = 234) of all participants responded correctly to all three presented results, with students having an overall higher result interpretation performance rate (63, 89%).Nine (3%) people failed in all three presented results (3 employees, 6 community members).Overall, 36 (12%) provided a false non-reactive result interpretation, 21 (7%) provided a false reactive result interpretation, and 14 (4%) provided both false nonreactive and false reactive result interpretations.Community members generally had a lower performance and students consistently performed best overall (Table 3).

Discussion
This HIVST performance evaluation among employees, students, and community members in Mozambique showed that there were some challenges in following the correct procedures.
Completing procedures incorrectly could have consequences such as invalid results and/or inaccurate interpretation.Important errors observed included incorrect sample collection procedures and not following the wait time instructions prior to result interpretation.Similar difficulties in performance using this same HIVST were seen in South-Africa: namely, removing the test tube, difficulty in placing the test tube in the stand, touching the flat pad, as well as difficulties in sample collection [5,6].However, despite the procedural errors, the overall usability in our study was quite favorable (80%).Other research groups have utilized different HIVST test kits, and slightly different questionnaires where the main steps were similarly observed but having subtle differences in a few of the individual steps, resulting in a usability calculation of 10 to 13 steps [5,6,11,21,22], while others only used main steps for the usability calculation [7,8].Overall, the usability index was high in these studies (reported scores between 85%-99%) [5,6,11,21], with some studies not providing a UI score but still reporting the proportion of errors during HIVST procedures (ranging between 15-30%) [7,8,22].We observed that many participants in our study did not read (or did not fully read) the instructions.The authors suspect that participants were in a hurry and wanted to spend as little time as possible on the study activities.It would be important to further understand if there are language or comprehension challenges that need to be addressed and included in communication and/or education strategies when promoting HIVST.On the other hand, continuous educational sessions and incorporation of the use of social media to regularly explain procedures might enhance knowledge.Educational sessions on HIVST procedures could be incorporated into community outreach activities and included in health promotion campaigns, for example, during community HIV counseling and testing, and index-case testing activities.Promotional videos or messages could be created which include information on HIVST procedures, including alerts on the main issues that might come up when performing the test such as the required waiting time for test results, handling of test materials (specifically, not touching the swab), and the need for test result confirmation.A false non-reactive result interpretation was given by 12% of the participants, which (if interpreting one's own results) could place the person at risk for not properly identifying a reactive HIVST result, and for missing the need for confirmatory testing and/or initiating necessary care and treatment.The proportion of false non-reactive result interpretation was higher among community members, potentially putting sexual partners at risk when available HIV prevention methods are not used.False reactive result interpretation was also more frequent among community members than other participant groups, though the occurrence was less frequent than for false non-reactive results.However, if effective linkage mechanisms are put in place and adhered to by HIVST users, a confirmation test can easily correct this false reactive interpretation error.
The MOH adopted a national community-based HIV self-testing strategy as of December 2021.Individuals can receive an HIVST and perform it at home or in assisted fashion with a counselor, whatever the individual prefers, but all reactive results need to be confirmed at the health facility, in accordance to existing national HIV counseling and testing guidelines [23].Our results showed that continuous demonstration of HIVST procedures through different formats (e.g., written instructions, video demonstration, community education sessions) is necessary to maintain quality performance of test procedures.Regular performance evaluations such as the one in our study could be integrated into training and longitudinal quality assurance activities (usually organized by Ministry of Health) to ensure proper HIVST result interpretation.Social media platforms and hotlines are other viable sources of communication for such messaging, as well as information obtained through community-or health facilitybased talks.
Limitations of the research included the restricted location of the study (only two rural districts in one province).Consequently, the results are not necessarily generalizable for the entire country.It is possible that the Hawthorne effect could have influenced participant behavior during the observations, resulting in overperformance.In addition, the manufacturers' instructions are only available in Portuguese, which could have influenced HIVST performance for participants who do not speak and/or are not literate in Portuguese, despite containing clear illustrations.

Conclusions
The study showed that usability of HIVST in rural Mozambique was favorable overall, despite some procedural errors, confirming results of other studies.Regular assessments using these validated instruments can ensure monitoring of HIVST performance quality.In addition, continuous communication and education strategies in the community and workplace seem to be necessary to ensure correct interpretation and to avoid missed opportunities for linkage to services.

Fig 1 .
Fig 1. Flow chart: Number of individuals recruited, eligible and consented for study participation.https://doi.org/10.1371/journal.pone.0305391.g001 de Sau ´de" (INS) (CIBS-INS, reference 080/CIBS-INS/2018), the Vanderbilt University Medical Center (VUMC) Institutional Review Board (IRB) (#181834), and the Mozambique National Directorate of Pharmacies, and was reviewed in accordance with the U.S. Centers for Disease Control and Prevention (CDC) human research protection procedures and was determined to be research.The CDC co-investigators did not interact with human subjects or have access to identifiable data or specimens for research purposes.Informed consent was administered by research assistants speaking the local language(s).All participants provided written informed consent prior to study participation.Additional information regarding the ethical, cultural, and scientific considerations specific to inclusivity in global research is included in the Supporting information (S1 File).

Table 1 .
(Continued) Chi-square test (or Fisher exact test when the frequency for certain category was less than 5) for the categorical variables; ANOVA for the continuous variables. α

put the tube correctly in the support (45-degree angle)? (Y) 49 (44.5%) 46 (64.8%) 63 (48.8%) 158 (51.0%)
Percentages of expected responses for each activity indicated the usability level for that activity.All percentages were averaged within each group to obtain the general UI for that group.

Table 3 . Interpretation results of pre-defined HIVST results, per priority population. Employees (n = 110) Students (n = 71) Community Members (n = 131) Total (n = 312) p-value α
False non-reactive interpretation was defined as an interpretation given as non-reactive while the test result was actually reactive or invalid γ False reactive interpretation was defined as an interpretation given as reactive while the test result was actually non-reactive or invalid https://doi.org/10.1371/journal.pone.0305391.t003 α Fisher exact test β